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Abstract 

Planning and executing volitional actions in the face of conflicting habitual responses is a crit- 
ical aspect of human behavior. At the core of the interplay between these two control systems lies 
an override mechanism that can suppress the habitual action selection process and allow execu- 
tive control to take over. Here, we construct a neural circuit model informed by behavioral and 
electrophysiological data collected on various response inhibition paradigms. This model extends 
a well established model of action selection in the basal ganglia by including a frontal executive 
control network which integrates information about sensory input and task rules to facilitate well- 
informed decision making via the oculomotor system. Our simulations of the antisaccade, Simon 
and saccadc-ovorridc task ensue in conflict between a prepotent and controlled response which 
causes the network to pause action selection via projections to the subthalamic nucleus. Our 
model reproduces key behavioral and electrophysiological patterns and their sensitivity to lesions 
and pharmacological manipulations. Finally, we show how this network can be extended to in- 
clude the inferior frontal cortex to simulate key qualitative patterns of global response inhibition 
demands as required in the stop-signal task. 

Download the model at: http://ski.clps.brown.eclu/BG_Projects/ 
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1 Introduction 



"Before you act, listen. Before you react, think. Before you spend, earn. Before your criticize, wait." 
This quote by Ernest Hemingway highlights our basic tendency to act impulsively while reminding 
us that sometimes it is advisable to inhibit these prepotent response biases and act more thoughtful. 
Recent scientific advancements have shed light on the neural and cognitive mechanisms that imple- 



ment inhibitory control of prepotent response biases ( Andres , 2003 Aron 2007 Logan 1985 Miyake 



et al. 


2000 


Stuphorn and Schall 


2006 


Munoz and Everhng 2004 



of tasks exist to study response inhibition empirically. Among the tasks thought to require selective 
response inhibition are the antisaccade task, the Simon task, and the saccade-override task. Each of 
these tasks induces a prepotent response bias that sometimes needs to be overridden with a controlled 
response based on executive control. For example, the antisaccade task requires subjects to saccade in 
the opposite direction of an appearing stimulus. The Simon task requires subjects to respond according 
to an arbitrary stimulus- response rule (e.g., respond left or right depending on stimulus color), but 
where the stimulus is presented on one side of the screen, inducing a prepotent response bias to that 
side. In congruent trials the stimulus is presented on the same side as the correct response indicated 
by the rule, whereas on incongruent trials it is on the opposite side. Finally, the saccade-override task 



(Isoda and Hikosaka 2007) requires subjects to saccade in the direction of a stimulus of a particular 
color for several repetitions in a row. On so-called switch-trials the instruction cue indicates that the 
other colored stimulus is now the target, so that the participant has to override the initial planned 
response and switch to the other one. While critical differences exist, all of these tasks require subjects 
to inhibit a prepotent response and replace it with a different response. In contrast, while also requir- 
ing response inhibition, the well-studied stop-signal task does not require subsequent initiation of an 



active response but only outright inhibition of the planned response ( Verbruggen and Logan 2008 ) 



Electrophysiological and functional imaging data implicate key nodes in frontostriatal circuitry as 
being active during response inhibition and executive control. At the cortical level, these include the 
right inferior frontal gyrus (rIFG) ( |Aron et al.[ 2003| Chambers et al. 2007 Sakagami et al. 2001 Xue 
et al.[ |2008p the dorsolateral prefrontal cortex (DLPFC) ( [Wegener et al.[ |2008[ [Funahashi et al.[ |1993 



Johnston and Everling 2006), the supplementary eye fields (SEE) (Schlag-Rey et al. 1997), the pre 



supplementory motor area (pre-SMA) ( Congdon et al. 2009[ [Aron et"aL 2007a| Isoda and Hikosaka 



20071, and the frontal eye fields (FEE) (Munoz and Everhng 2004 1. At the subcortical level, the 



striatum (Zandbelt and Vink 2010[ Watanabe and Munoz 2011 Ford and Everling 2009), the sub 



thalamic nucleus (STN) ( [Eagle et al. 2008 Isoda and Hikosaka 2008 Hikosaka and Isoda 2008 Aron 



and Poldrack 2006 Aron et al. 2007a) and the superior colliculus are involved. Manipulations that 



disrupt processing in either frontal or subcortical areas cause deficits in response inhibition (Cham- 



bers et al. 2007 Ray et al. 2009 Verbruggen et al. 2010). Moreover, response inhibition deficits are 



commonly observed in a wide range of psychiatric patients with frontostriatal dysregulation, including 



2001 


Oosterlaan et al. 


1998 Schachar and 





Logan[ 1990), obsessive compulsive disorder (OCD) (Chamberlain et al. 2006 Menzies et al. 2007 



Penades et"al| |2007[ [Morein-Zamir et al.[ |2009| ), schizophrenia (SZ) ( [Huddy et aTj |2009[ [Bellgrove 



et al. 2006 Badcock et al. 2002), Parkinson's disease (PD) (van Koningsbruggen et al. 2009) and 



substance abuse disorders (Monterosso et al. 2005 Nigg et al. 2006). 



Together, the above data suggest that intact functioning of the entire fronto-basal ganglia network 
is required to support response inhibition. However, it is far from clear that the underlying source of 
these deficits is the same. Inhibitory control is a very dynamic process, influenced by different inter- 
acting cognitive variables and neuromodulatory systems. Thus, response inhibition can be impacted 
by not only dysfunctional stopping per se, but can also be influenced by changes in motivational state 



(Leotti and Wager 2010), attentional saliency (Morein-Zamir and Kingstone 2006), maintenance and 



retrieval of task rules ( Hutton and Ettinger 2006 Nieuwenhuis et al. , 2004 Renter and Kathmann 



2004 Roberts et al. 1994), and separable modulations of selective vs global inhibition mechanisms 



(Aron 2011), to name a few. Although electrophysiological recording studies demonstrate neuronal 



populations that differentiate between successful and unsuccessful stopping ( Isoda and Hikosaka 2008 



2007), or inhibition of prepotent responses in favor of controlled responses (Watanabe and Munoz 2009 



Ford and Everling 2009 ) , there is at present no coherent framework integrating all of these findings 



into a single model attempting to account for patterns of electrophysiological data, or selective disrup- 
tions of component parts and their effects on behavior. 



The point of departure for our neural model builds on existing theorizing and data regarding the 
differential roles of the three main pathways linking frontal cortex with the basal ganglia (BG), often 
referred to as the direct, indirect and hyperdirect pathways. According to this framework, the corticos- 
triatal direct "Go" and indirect "NoGo" pathways together implement a selective gating mechanism 
by computing the evidence for facilitating or suppressing each of the candidate motor actions identi- 
fied by frontal cortex. Dopamine plays a critical role in this model by differentially modulating the 
activity levels in the two striatal populations, affecting both learning and choice. During rewards and 
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punishments, phasic bursts and dips in dopamine neurons convey reward prediction errors (Montague 



et ah 1997) that transiently amphfy Go or NoGo activity states, and therefore activity-dependent 



plasticity. In this manner, these striatal populations learn the positive and negative evidence for each 



cortical action ( Frank 2005 1 . More chronic increases in tonic dopamine levels also directly affect 
choice by shifting the overall balance of activity toward the Go pathway over the NoGo pathway, 
thereby emphasizing learned positive relative to negative associations and speeding responding (and 
vice- versa for tonic decreases in dopamine). Many of this model's predictions have been validated 
with behavioral studies involving dopaminergic manipulations and functional imaging in humans and 



monkeys (e.g., Frank et al. 2004 Nakamura and Hikosaka 2006 Palminteri et al. 2009 Voon et al. 



2010 Jocham et al. 2011 ), and synaptic plasticity and opto-genetic and genetic engineering studies in 



rodents (Kravitz et al. 2010 Hikida et al. 2010 Shen et al. 2008 Kravitz et al. 2012 1. 



Note that in the above model, responses are selectively facilitated or suppressed via separate stri- 
atal Go and NoGo populations modulating the selection of particular cortical actions. However, more 
recent models have also incorporated the third hyperdirect pathway from frontal cortex to the STN to 
BG output. Communication along this pathway provides a global and dynamic regulation of the gating 
threshold, by transiently suppressing the gating of all responses when there is conflict between alter- 



native actions (Frank 2006 Ratcliff and Frank, 2012). Empirical studies using STN manipulations 



(Frank etal. 2007a Wylie et al. 2010 Cavanagh et al.[ 20TT] ) direct recordings (Cavanagh et al. 2011 



Isoda and Hikosaka 2008 Zaghloul et al. 2012), and fMRI/DTI (Aron et al. 2007a) have similarly 



supported this notion. 



Nevertheless, the existing BG model cannot handle situations in which an initial prepotent re- 
sponse is activated but then needs to be suppressed - either altogether, or in favor of a more controlled 
response - situations typically studied under the rubric of "response inhibition" . Here, we extend 
the model by incorporating additional cortical regions that facilitate executive control and can inhibit 
and override the more habitual response selection mechanism. We consider dynamics of the prepotent 
response process, the subsequent detection that this response needs to be inhibited, and the inhibition 
process itself ~ and how all of these factors are modulated by biological and cognitive variables. We 
consider electrophysiological data in various frontal (DLPFC, FEF, preSMA, ACC) and basal ganglia 
(striatum, STN) regions that are well captured by the model, and how these are linked to functional 
parameters of a high level decision making process embodied by a variant of the drift diffusion model. 
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Neural models are complex, in that they involve a number of parameters interacting to produce 
nonlinear effects on dynamics and behavior. There is also a risk of overfitting that could result from 
adjusting parameters to precisely match electrophysiological data from one experiment, which may 
make it difficult to precisely capture electrophysiological (or behavioral) data from a different exper- 
iment. Thus our aim was instead to capture qualitative patterns of data in both electrophysiology 
at multiple levels of cortical and subcortical network, and of the effects of their manipulation on be- 
havior, with a single set of parameters]^ In other work (Wiecki & Frank, in preparation) we show 
that systematic variations in neural model parameters are related in a lawful, monotonic fashion to 
more computational level parameters in a modified drift diffusion framework, providing a principled 
understanding and falsifiable experimental predictions. Moreover, despite the qualitative nature of 
model fits here, we nevertheless aim to distinguish our model from others in the literature based on 
general principles independent of particular parameterizations. Towards this goal we extracted a set 
of qualitative behavioral and neurocognitive benchmark results (listed in the results section) which we 
use to assess the validity of our model and compare to other models. 

As noted above, despite surface features suggesting a single integrated response inhibition network, 
there are actually multiple dynamic components that can affect inhibition. Our contribution in this 
paper is to formalize these separable neural processes, to explore their interactive dynamics. To 
summarize and preview the core aspects of our work: 

• We present a neural network model of the three main frontal-BG pathways supporting prepotent 
action selection, inhibitory control, conflict-induced slowing, and volitional action generation. 

• We show that behavioral changes in a range of tasks dependent on these basic processes can 
result from alterations in brain connectivity and state and provide testable predictions for effects 
of distinct brain disorders. 

• Selective response inhibition involves global conflict-induced slowing via the hyperdirect pathway, 
raising the effective decision threshold to prevent prepotent responding, followed by DLPFC 
induction of striatal NoGo activity to inhibit the planned prepotent response. Subequently, 
the DLPFC provides top-down facilitation onto striatal Go populations encoding the controlled 
response. 

^By qualitative we moan that we do not attempt to quantitatively fit the precise shape of firing of any given cell 
type, but we do aim to show that a given population of cells increases or decreases firing rate at a particular point in 
time relative to some task event or to some estimated cognitive process. For example, for an area to be involved in 
inhibition it must show increased activity prior to the time it takes to inhibit a response. Or in striatum, particular 
cell populations are active related to biasing the prepotent response, suppressing that response, and then activating the 
controlled response - our model recapitulates this qualitative pattern. 
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• Response selection and inhibition are further regulated by neuromodulatory influences including 
dopamine linked to changes in motivational and attentional state. Dopamine reflects potential 
reward values and facilitates Go actions. In addition, our model suggests that while selective 
response inhibition is influenced by tonic levels of DA, global response inhibition is not. 

• Our model is challenged in its ability to overcome prepotent responses and evaluated by its ability 
to reproduce key qualitative patterns reported in the literature, including: 

— Behavioral RT distribution patterns in selective response inhibition tasks. 



Electrophysiological activity patterns of the FEF (Everling and Munoz 2000), pre-SMA 



(Hikosaka and Isoda 20081, the STN (Isoda and Hikosaka 2008), striatum (Watanabe and 



Munoz 2009), SC (Pouget et al. 2011 Pare and Hanes 2003) and scalp recordings (|Yeung 



et al. 2004a). 



— Psychiatric, developmental, lesion and pharmacological manipulations of frontal function 
and DA modulations. 

• We show that when our model is extended to include the rIFG it can recover key electrophysio- 
logical and behavioral data from the stop-signal task literature. 

In sum, this approach provides a mechanistic account of a major facet of cognitive control and 
executive functioning, which we hope will allow for a richer understanding of the relationship between 
behavioral, imaging, and patient findings. 

2 Neural Network Model 

We first introduce the neural circuit model of interacting dynamics among multiple frontal and basal 
ganglia nodes and their modulations by dopamine. We then describe how we vary model parameters 
to capture biological and cognitive manipulations. 



Overview The model is implemented in the Emergent software ( Aisa et al. 2008 ) with the neuronal 



parameters adjusted to approximate known physiological properties of the different areas (Frank 



2005[ 2006). The simulated neurons use a rate-code approximation of a leaky integrate-and-fire neu- 
ron (henceforth referred to as units) with specific channel conductances (excitatory, inhibitory and 
leak). Multiple units (simulated neurons) are grouped together into layers which correspond to dis- 
tinct anatomical regions of the brain. Units within each layer project to those in downstream areas, 
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and in some cases, when supported by the anatomy, there are bidirectional projections (e.g., bottom- up 
superior coUiculus projection to cortex as well as top-down projections from cortex to colliculus). We 
summarize the general functionality of the model here to foster an intuitive understanding; implemen- 
tational and mathematical details can be found in the appendix. While a single set of core parameters 
(i.e. integration dynamics and overall connection strength between layers) is used to simulate various 
electrophysiological and behavioral data in the intact state, each reported simulation is tested on 8 
networks with randomly initialized weights between individual neurons. The model can be downloaded 
from our online-repository http : / /ski . dps . brown . edu/BG_Pro j ects . 



The model represents an extension of our established model of the BG (Frank 2005 2006 Wiecki 



and Frank 2010). Because the extended model involves multiple components, we will progressively 



introduce each part, beginning with its core and then describing how each new component contributes 
additional functionality. 



Basic basal ganglia model The architecture of the core model is similar to Frank ( 2006 1 . While the 



original model simulated manual motor responses, our model features a slightly adapted architecture 
in accordance to the neuroanatomy and physiology underlying rapid eye-movements (i.e. saccades) as 
reviewed in Hikosaka (2007) and Munoz and Everling ( 2004| . Stimuli are presented to the network in 
the input layer, corresponding to high level sensory cortical representations. An arbitrary number of 
motor responses can be simulated, but here we include a model with just two candidate responses. The 
input layer projects directly to the cortical response units in the frontal eye fields (FEF) which imple- 
ments action planning and monitoring and projects to the superior colliculus (SC), which acts as an 



output for saccade generation (Sparks 2002). The SC consists of two units coding for a leftward and a 



rightward directed saccade. If the firing rate of one unit crosses a threshold, the corresponding saccade 



is initiated (Everling et al. 1999). The time it takes an SC unit to cross its threshold from trial onset 



is taken as the network's response time (RT). Stimulus-response mappings can be prepotently biased 
by changing projection strengths (i.e. weights) so that certain input patterns preferentially activate 
a set of FEF response units more than the alternative response units. (These sensory-motor cortical 
weights can also be learned from experience, such that they come to reflect the prior probability of 
selecting a particular response given the sensory stimulus; (Frank 2006[ )). In fact, with only these 
three structures our model would only be capable of prepotent, inflexible responding. 



By itself, FEF activation is not sufficiently strong to initiate saccade generation because the SC is 
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Figure 1: Box-and-arrow view of the neural network model. The sensory input layer projects to the 
FEF, striatum and executive control (i.e. DLPFC, SEF and pre-SMA). Via direct projections to FEF 
(i.e. cortico-cortical pathway), stimulus-response-mappings can become ingrained (habitualized) . FEF 
has excitatory projections to the SC output layer that executes saccades once a threshold is crossed. 
However, under baseline conditions, SC is inhibited by tonically active SNr units. Thus, for SC units 
to become excited, they have to be disinhibited via striatal direct pathway Go unit activation and 
subsequent inhibition of corresponding SNr units. Conversely, responses can be selectively suppressed 
by striatal NoGo activity, via indirect inhibitory projections from striatum to GP and then to SNr. 
Coactivation of mutually incompatible FEF response units leads to dACC activity (conflict or entropy 
in choices), which activates STN. This STN surge makes it more difficult to gate a response until the 
conflict is resolved, via excitatory projections to SNr, effectively raising the gating threshold. Striatum 
is innervated by DA from SNc which amplifies Go relative to NoGo activity in proportion to reward 
value and allows the system to learn which actions to gate and which to suppress. The instruction 
layer represents abstract task rule cues (e.g. antisaccade trial). The DLPFC integrates the task cue 
together with the sensory input (i.e. stimulus location) to initiate a controlled response corresponding 
to task rules, by activating the appropriate column of units in FEF and striatum. 
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under tonic inhibition from the BG output nucleus: the substantia nigra pars reticulata (SNr), whose 
neurons fire at high tonic rates. However, the tonic SNr-SC inhibition is removed following activation 
of corresponding direct (Go) pathway striatal units, which inhibit the SNr, and therefore disinhibit 
the SC ( |Hikosaka[ |1989[ [Hikosaka et al.[ |2000[ [Goldberg et al.[ |2012p . The indirect pathway acts in 
opposition to the direct pathway by further exciting the SNr (indirectly, via inhibitory projections to 
the globus pallidus (GP) which inhibits the SNr). Thus, direct pathway activity results in gating of 
a saccade (i.e. Go) while indirect pathway activity prevents gating (i.e. NoGo). Striking evidence for 
this classical model was recently presented by optogenetic stimulation selectively of direct or indirect 
pathways cells, showing inhibition or excitation of SNr respectively, and resulting in increased or de- 



creased movement (Kravitz et al. 2010). 



The Go and NoGo striatal populations include multiple units that code for the positive and negative 
evidence in favor of the FEF candidate actions given the sensory input context. Relative activity of the 
striatal pathways is modulated by dopaminergic innervation from the Substantia Nigra pars compacta 
(SNc) due to differential simulated Dl and D2 receptors present in the two pathways. In particular, 
dopamine further excites active Go units while inhibiting NoGo units. These effects on activity also 
produce changes in activity-dependent plasticity, allowing corticostriatal synaptic strength in the Go 
population to increase following phasic dopamine bursts during rewarding events, and those in the 



NoGo population to decrease (and vice- versa for negative events; (Frank 2005)). For simplicity, in 
the present model we omit learning because the paradigms we simulate do not involve learning, and 
focus on associations that have already been learned. However, it is now well known that striatal unit 
activity is modulated by the reward value of the candidate action, such that rewarding saccades are 



more likely to be disinhibited ( Hikosaka et al. 2006 ) 



Bottom-up projections from SC to FEF allow action-planning to be modulated according to direct 
and indirect pathway activity (Sommer and Wurtz 2006 2004a|b 2002). This effectively forms a 
closed loop in which FEF modulates the striatum which, via gating through SNr and SC, in turn 
modulates the FEF. Loosely, FEF considers the candidate responses and "asks" the BG if the corre- 
sponding action should be gated or not. Thus, with these structures the model can selectively gate 
responses modulated by DA. 



In addition to the above gating dynamics, the overall threshold for gating is controlled by the ease 
with which the SNr units are inhibited by the striatal Go units. The STN sends diffuse excitatory 
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projections to the SNr (Parent and Hazrati 1995), and therefore when STN units are active they 



increase the gating threshold for all responses, effectively contributing a 'global NoGo' signal (Frank 



2006 Ratcliff and Frank 2012). The STN does not however, act as a static increase in threshold. 
Rather, the STN receives input directly from frontal cortex, and becomes more active when there is 
response conflict (or choice entropy) during the early response selection process. In the current model, 
conflict is computed explicitly by the dorsal anterior cingulate cortex (dACC), which detects when 
multiple competing FEF response units are activated concurrently, and in turn activates the STN to 
make it more difficult to gate any response until this conflict is resolved. The full computational role 
of dACC is far from resolved and likely to be more complex than conflict detection and control (see. 



Holroyd and Coles 


2002 


Botvinick et al. 


2004 


Alexander and Brown 


2011 


Kolling et al. 


2012 



Nevertheless, alternative accounts of dACC function (Kolling et al. 2012 ) are entirely compatible with 
our model (an issue we return to in the discussion), but for convenience we label the computation as 
"conflict" . 



Ftontal Pathway model 

Volitional response selection So far our model is able to select /gate responses and slow down 
gating when an alternative response appears to have some value relative to the initial planned action. 



However, SRITs require executive control: integration of the sensory state together with the task 
rule to not only inhibit the prepotent response but replace it with a volitional one. Such rule-based 
processing is effortful and time-consuming, and hence the controlled response process lags that of the 
initial fast response capture. Based on a variety of evidence, we ascribe the rule-based representations 



to the dorso-lateral prefrontal cortex (DLPFC) (e.g. Miller and Cohen 2001 Chambers et al. 2009) 



This structure is involvedin the active maintenance of stimulus-response rule representations (Derrfuss 



et al. 


2004 


2005 


Brass et al. 


2005 


), is necessary for correct antisaccade trials ( 


Wegener et al. 


2008 



Funahashi et al. 1993 Johnston and Everling 20061, and is involved in selective response inhibition 



( Caravan et al. 2006 Simmonds et al. 2008 1 and response selection ( Braver et al. 2001 Rowe et al. 



2002 1. Moreover, SEF (Schlag-Rey et al. 1997) and pre-SMA (Isoda and Hikosaka 2007 Ridderinkhof 



et al. 2011) are also critically involved in correct SRIT performance. 



We consequently added an abstract executive control layer to summarize the DLPFC, SEF and 
pre-SMA complex (in the future referred to as DLPFC). This layer selects FEF responses and biases 
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Figure 2: Neural network model in different task conditions, a) Prosaccade condition. (1) Left 
stimulus is presented in input layer; (2) Prepotent weights bias left response coding units in FEF; (3) 
Left response Go gating neurons in striatum are activated; (4) Left response coding units in SNr are 
inhibited; (5) The left response unit in SC is disinhibited, and due to recurrent excitatory projections 
with FEF, is excited and the action is executed, b) Antisaccade condition. The activity pattern early 
in the trial (i.e. before DLPFC comes online) is similar to that in the prosaccade condition. (1) Left 
stimulus is presented in input layer activating prepotent left response in FEF; (2) The unit coding for 
the antisaccade condition is externally activated in instruction layer; (3) DLPFC integrates sensory 
and instruction input according to task rules and activates right coding units in FEF together with 
right Go gating units left NoGo units in striatum; (4) in FEF, right coding units are activated due 
to DLPFC input in addition to the prepotent left coding units already active; (5) dACC detects co- 
activation of multiple FEF action plans and activates (6) hyperdirect pathway to excite STN and SNr, 
globally preventing gating until conflict is resolved. Eventually, stronger controlled DLPFC activation 
of the right coding FEF response results in gating of the correct antisaccade (7). In some trials, DLPFC 
activation is too late and the prepotent left saccade will have already crossed threshold, resulting in 
an error. 
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BG gating according to task rules (see figure [ij. Although not explicitly represented separately in the 
model architecture, we conceptualize the individual contribution of DLPFC as rule encoding and ab- 
stract action selection whereas SEF and pre-SMA are transforming this abstract action representation 



into concrete motor-actions (Schlag-Rey and Schlag 1984 Schlag and Schlag-Rey 1987 Curtis and 



DEsposito 2003). In turn, these planned motor actions can influence the selected response in FEF 



and bias gating via projections to striatal Go and NoGo neurons (Munoz and Everling 2004) 



Anatomical and functional studies demonstrate projections from both DLPFC to SEF and pre- 



SMA (Lu et al. 1994 Wang et al. 2005 ) and to striatum to affect response gating ( Haber 2003 Doll 



et al. 2009 Frank and Badre 2011); and from SEF to FEF (Huerta et al. 19871. We explore how 



these projections impact dynamics of response selection. But how does the executive controller in our 
model 'know' which rule to activate? We do not address here how these rule representations arise via 



learning, which is the focus of other PFC-BG modeling studies (see Rougier et al. , 2005 Frank and 



Badre, 2011 Collins and Frank 2012). Instead, we simulate the state of the network after learning 



by simply including an Instruction layer as a second input layer to the model encoding task condition 
(e.g. antisaccade trial). In case of the antisaccade task, the sensory input layer encodes the direction 
of the visual stimulus and the instruction layer encodes whether the network should perform a pro 
or antisaccade. The DLPFC complex then integrates these two inputs and activates a (pre-specified) 
rule unit that (i) projects to the correct FEF response units supporting the antisaccade; (ii) activates 
striatal NoGo units to prevent gating of the active prepotent pro-saccade response, and (iii) activates 
striatal Go units encoding the controlled antisaccade. 

Critically, DLPFC units are relatively slow to activate the appropriate rule unit. This is due to the 
need to formulate a conjunctive rule representation between the visual location of the stimulus and the 
task instruction (either one of these is not sufficient to determine the correct response, and indeed, each 
individual input provides evidence for multiple potential rules). Time constants of membrane potential 
updating is reduced to support this integration, which also is intended to approximate slower time 
course of rule retrieval and subsequent computation to determine the correct action (via interactions 
with preSMA and SEF). Moreover, we include considerable inter-trial noise in DLPFC activation 
dynamics so that executive control is available earlier on some trials while later on others. The slowed 
integration and the increase of inter-trial noise in executive control are necessary for the model to 
capture the quantitative benchmark results (demonstrated below). Moreover, the slower controlled 



processing is also a core feature of classical dual process models of cognition (e.g. Sloman 1996 ) and 
the increased noise accords with the general statistical observation that longer latencies are typically 
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associated with greater variability. 



Competition between the two response selection mechanisms As outlined above, our model 
features two response selection mechanisms: (i) a fast, prepotent mechanism driven by a biased pro- 
jection from sensory input to FEF; and (ii) a slow, volitional mechanism that originates in the DLPFC 
which integrates instruction and sensory input to select and gate the correct response. Importantly, 
the volitional mechanism is slower but stronger than the prepotent one. If, due to noise in the speed of 
integration, executive control is slower on some trials, it might be too late to activate the correct rule 
representation before the prepotent response is gated. In contrast, when the executive controller is 
faster, it activates the alternative FEF response, leading to conflict-induced slowing, and then actively 
suppresses the prepotent response via projections to striatal NoGo units encoding the prosaccade. 
This conceptualization can be regarded as a biologically plausible implementation of the cognitive 



activation-suppression model (Ridderinkhof 2002 Ridderinkhof et al. 2004). Note however that our 



implementation involves two suppression mechanisms, one in which conflict results in global threshold 
adjustment, and another in which the prepotent response is selectively inhibited. 

Modulations To test the influence of different biological manipulations on executive control paradigms, 
we modify various parameters in the network model. Here, we list the different modulations and their 
implementation. 

• Prepotency: To simulate differences in the strength of the prepotent response capture of an 
appearing stimulus (e.g., the prosaccade stimulus) we modulate the projection strength between 
sensory input to the dominant response units in FEF and striatum. 

• Speed of DLPFC: To simulate efficacy of prefrontal function we modulate the speed of DLPFC 
integration, by adjusting the time constant of membrane potential updating in these units. Faster 
updating implies proactive control. 

• Connectivity of DLPFC : To simulate differences in intra-cortical connectivity we modulate the 
DLPFC-l^FEF projection strength. 

• Speed-accuracy trade-off: To simulate strategic adjustments in the speed-accuracy trade-off, we 



modulate the connection strength between frontal cortex and striatum (Forstmann et al. 2010). 
In particular, when speed is emphasized, the FEF more effectively activates striatal Go units 
so that it is easier to reach gating threshold. In contrast, accuracy adjustments are reflected in 
increased STN baseline ultimately increasing the response gating threshold. 
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• STN impact: STN contributions are simulated by manipulating the relative synaptic strengths 
from STN to SNr, effectively changing the amount of STN activity required to prevent BG gating 



(Ratcliff and Frank 2012 Cavanagh et al. 20111 



• tonic DA: Pharmacological and disease modulations of DA levels are simulated by either de- 
creasing (e.g., PD) or increasing (e.g., SZ) tonic DA activity, which in turn modulates relative 
activity of Go vs NoGo units. 

2.1 Selective Response Inhibition 
2.1.1 Methods 

As summarized earlier, all SRITs have a common task structure, (i) A prepotent response bias is 
induced by priming an action. In the antisaccade task this is a result of the appearance of a stimulus 



that initiates a 'visual grasping reflex' (Hess et al. 1946); in the Simon task this is the result of 



placing the target stimuli on either side of the screen, initiating a response capture ( Ridderinkhof 



2002 1 ; in the saccade-overriding task this is the result of repeated responding to the same colored 
stimulus which renders this response habitual, (ii) In congruent trials, the correct response is the 
same as the prepotently biased one. (iii) In incongruent trials, the correct response is incompatible 
with the prepotently biased response, and subjects can use executive control to suppress the initially 
predominant action in favor of the task-appropriate one. 

We implemented this common task structure as follows in our neural network model (alternative 
task implementations that accommodate the differences between the tasks lead to similar patterns so 
we simplified in order to use a single task representation of this basic process, but nevertheless simulate 
patterns of data evident in specific tasks below). Two stimulus positions, left and right, were encoded 
in the input layer as two distinct columns of activated units. The prepotent bias toward an appearing 
target was hard-coded by strong weights from each input stimulus to corresponding response units in 
FEE. This prepotent weight facilitates fast responding for congruent trials, but biases responding in the 
erroneous direction for incongruent trials. The DLPEC layer integrates sensory input and instruction 
input to activate a conjunctive rule unit encoding the unique combination of sensory and instruction 
input, which then projects to the associated correct response unit in FEE. Each of the four DLPEC 
units project to the appropriate FEE response unit. Note that weights from the DLPFC to the FEE 
are stronger than the prepotent bias connection from the input layer to the EEF so that the DLPEC 
would eventually override an erroneous prepotent response. (The same functionality could be achieved 
by simply allowing DLPEC units to reach a higher firing rate or to engage a larger population of units. 
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instead of adjusting the weights). In addition, DLPFC units activate corresponding Go and NoGo 
units in the striatum (e.g. in an antisaccade trial, Go units coding for the correct response and NoGo 
units coding for the incorrect response get activated by top-down PFC input). 



2.1.2 Results 

We identified a set of key behavioral and neurophysiological qualitative patterns across SRITs that 
form desiderata for our model to capture: 



^1 Incongruent trials are associated with higher error rates than congruent trials (e.g Reilly et al. 



2006 McDowell et al. 2002 Isoda and Hikosaka 2008 ) 



#2 Reaction times (RTs) are faster for errors than correct trials (e.g Reilly et al. 2006 McDowell 



et al. 2002 Isoda and Hikosaka 2008 ) 



#3 Strategic adjustments in the speed-accuracy trade-off (via changes in decision threshold) mod- 



ulates functional connection strength between frontal cortex and striatum (Forstmann et al. 



2010). Similarly, STN activity is associated with modulations of the decision threshold (Ratcliff 



and Frank 2012 Cavanagh et al. 2011). 



#4 Various psychiatric diseases associated with frontostriatal cathecholamine dysregulation lead to 



Reilly et al. 


2006 


Harris et al. 


2006 


Reilly 



et al. 2007 McDowell et al. 2002). 



#5 Early activation of prepotent motor response, e.g. in EMG measurements (Burle et al. 2002) 



#6 At least four different types of activation dynamics in FEF neurons during correct and error 



incongruent trials ( Everling and Munoz 2000 1 . Specifically, neurons coding for the erroneous 



(i.e. prepotent) response are fast to activate and their activity is greater on error trials than 
correct trials. In contrast, neurons coding for the correct (i.e. controlled) response are slower to 
activate and their activity is reduced and delayed on error trials. See figure|6]; for the quantitative 
data that forms the basis of this qualitative pattern. 

#7 At least four different types of striatal neurons with dissociable dynamics and direction selectivity 



in congruent and incongruent trials (Watanabe and Munoz 2009 Ford and Everling 2009) 



Specifically, (i) during prosaccades, distinct neural populations code for facilitation of the correct 
response and suppression of the alternative; (ii) during antisaccade trials, (iia) neurons coding 
for facilitation of the incorrect prepotent response initially become active but return to baseline 
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Figure 3: a) Error rates in incongruent trials ± SEM relative to intact networks for different neural 
manipulations. Networks make more errors with increased tonic DA levels, or STN dysfunction, 
compared to intact networks, b) Response Times (RTs) ± SEM relative to intact networks, for pro 
and antisaccade trials as a function of neural manipulations. For more analysis see the main text. 

when (lib) neurons coding for the suppression of that response become active together with (iic) 
neurons coding for facilitation of the correct controlled response (see figure [9]: 



Neurons forming part of the hyperdirect pathway from frontal cortex (pre-SMA, dACC) to the 
STN show increased activity (i) before correct incongruent responses and (ii) after incorrect 



incongruent responses, but (iii) baseline activity during congruent response (Isoda and Hikosaka 



2007[ |2008| |Yeung et aLj |2004a| [Zaghloul et al.[|2012[ ). This pattern of activity co-occurs with 



delayed but more accurate incongruent responding. 

In the following, we demonstrate how our model reproduces these qualitative patterns, before 
linking its dynamics to a higher level computational description. 

Behavior As expected, intact networks make considerably more errors on incongruent trials (error 
rate of 15%) as compared to perfect performance in congruent trials (error rate close to 0%, not shown), 
thereby capturing qualitative pattern |^1[ 

Further, networks in general have longer response times (RTs) in incongruent trials (see figure 
|3(b)| thus capturing qualitative pattern jf2 Incongruent trials are slower for two reasons: (i) it takes 



time for executive control (DLPFC) computations due to the requirement to integrate two sources of 
input to activate the associated rule; and (ii) once activated, the controlled response confiicts with the 
prepotent response, leading to STN activation and associated increases in BG gating threshold. 
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Additional analysis revealed that incongruent error trials are associated with faster RTs compared 
to correctly performed incongruent trials (figure |4]). In our model, errors are made when the faster 
prepotent action reaches threshold before the inhibitory process can cancel it. This mechanism allows 
the model to capture qualitative pattern |^2| and |^3| 



We next investigated how these behavioral patterns were affected by manipulations (see figure 3(a) ). 
Incongruent error rates were most exaggerated with increased tonic DA levels, and by disrupted STN 
function to simulate deep brain stimulation. The effect of increased striatal DA on incongruent error 



rates captures corresponding patterns (see #4) observed in non-medicated schizophrenia patients, who 



Reilly et al. 


2006 


Harris et al. 


2006 


Reilly et al. 


2007 


McDowell et al. 



2002 1 . Tonic DA elevations are associated with speeded responding in both congruent and incongruent 



trials, due to shifted balance toward the Go pathway facilitating response gating. This same mech- 
anism explains the increased antisaccade error rate. Conversely, decreased tonic DA leads to slowed 
responding due to increased excitability of the indirect NoGo pathway. The model also predicts that 
STN dysfunction produces increased error rates, due to an inability to raise the threshold required for 
striatal facilitation of prepotent responses. Indeed, STN-DBS induces impulsive (fast but inaccurate) 



responding in SRITs (Wylie et al. 2010). 



Finally, we tested in more detail how systematic parametric changes in a biological variable af- 



fect RT and accuracy. Figure 5(a) shows how RT distributions change under different settings of 



FEF— 7>striatum connection strength. Figure 5(b) shows quantitatively how increases in FEF->striatum 



connectivity leads to faster RT and decreased accuracy (qualitative pattern 7^31. Loosely, increasing 
FEF connection strength onto Go-units in the direct pathway leads to faster gating of responses. Con- 



versely, increases in STN— >SNr connectivity lead to slower RT and improved accuracy (figure 5(c)). 
The reason for both of these effects is that they differentially modulate SNr activity. Recall that the 
SNr tonically inhibits the thalamus, unless it is itself inhibited by the striatal direct pathway. Hence 
any modulation of the ease with which SNr units are inhibited - either via stronger connections from 
cortex onto Go units, or by increasing the SNr via the STN - will change the threshold required for 



the BG to gate an action. Indeed, RatclifF and Frank (2012) and Lo and Wang (2006) have shown 
that these two mechanisms are related to changes in the decision threshold in sequential sampling 
models. Our model subsumes both of these mechanisms, and suggests that these different routes are 
themselves modulated by distinct cognitive variables, such as volitional speed-accuracy modulation 
and conflict /choice entropy (cortico-striatal and STN). We return to this issue in the Discussion. 
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Figure 4: a) RT histogram for correct and erroneous incongruent trials in the model. Error RT 
distributions were shifted to the left due to prepotent response capture. This pattern is exaggerated 
with increased tonic DA due to lowered effective gating threshold, b) RT histograms of a monkey 
during the switch-task (reproduced from [Isoda and Hikosaka (2008 )). In blocks of trials, monkeys 
are continuously rewarded following saccades to one of two targets. On so-called 'switch-trials' a cue 
indicates that the monkey should perform a saccade to the opposite target, requiring the monkey to 
inhibit his planned saccade and perform a saccade to the opposite direction. As in the model, errors 
are associated with shorter reaction time, c) Reaction time distribution of an alternative model with 
fast DLPFC integration speeds. Correct trials are in red and errors in gray (not present). This model 
cannot account for the behavioral pattern of errors and RTs as a function of congruency, in contrast 
to models with slowed DLPFC integration (panel a). 



In sum, our model captures key qualitative behavioral patterns described in the literature (see 
above). Moreover, these patterns hold over varying biologically plausible parameter ranges leading 
to predictable changes in the behavioral patterns. However, given the complexity of the underlying 
model, it is also important to establish whether the internal dynamics of the different nodes of the 
network are consistent with available electrophysiological data in this class of tasks. 

Neurophysiology 

DLPFC, SEE and pre-SMA activity Our model summarizes the computations of the execu- 
tive control complex as a single layer corresponding to DLPFC, SEF and pre-SMA. One of our central 
predictions is that DLPFC activation must be delayed relative to the habitual response mechanism 
in order to produce the desired qualitative patterns. To demonstrate the plausibility of this account 
we simulated networks with increased DLPFC speed (time constant of membrane potential updating) . 
Consequently, networks ceased to make fast errors while correct RTs became much faster and more 
peaked (figure |4j:). The reason for this pattern is that active executive control now dominates and 
overrides the prepotent mechanism during early processing. This result implies that some delay in 
executive control is needed to account for empirical findings in which incongruent RTs are delayed. 
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Figure 5: a) RT distributions for incongruent trials by network models. FEF— 7>striatuni projection 
strengths were varied along the x-axis. Correct RT distributions are on the right side of each panel and 
incorrect RT distributions are on the left side, mirrored on the y-axis. This manipulation is equivalent 
to a speed-accuracy adjustment, as shown empirically to vary with pre-SMA— >-striatal communication 
(Forstmann et al, 2008; 2010), where here FEF plays the role of pre-SMA for eye movements as 
compared to manual movements studied in Forstmann et al. b+c) Speed-accuracy tradeoff under 
parametric modulation of (b) FEF— >-striatum connection strength and (c) STN— >SNr connection 
strength (color coded). Black represents low and yellow high connection strength. This pattern is 
consistent with decision threshold modulation. The absolute values of connection strengths in these 
different routes are chosen to lie on a sensitive range producing observable effects for demonstration 
purposes. 
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Figure 6: a) Average activity of individual superior colliculus (SC) units coding for the correct and 
error response in correct and incorrect trials during incongruent trials aligned to stimulus onset. The 
prepotent (i.e. erroneous) response comes on before the volitional, correct response. In incorrect trials 
the error-unit threshold is crossed before the volitional response unit gets active. In correct trials 
the error-unit is inhibited in time, b) Average activity of individual FEF units coding for prepotent 
error responses and volitional correct responses during incongruent trials aligned to stimulus onset 



(benchmark pattern ^6 ) c) Electrophsyiological recordings in FEF of monkeys (reproduced from 
Everling and Munoz| pOOO[)). 



SC and FEF activity Comparing single unit activation patterns of SC (see figure |6p,) to those 
of FEF (see figure|6jD) reveals that the activation dynamics are very similar between those two regions. 
Our model thus predicts that FEF can be interpreted as a cortical saccade planning/monitoring area 
that directly influences saccade generation via its projections to SC ( Munoz and Everling[ 2004). 
Moreover, SC activity reveals that in both, correct and incorrect incongruent trials, the incorrect 
prepotent response unit becomes active before the controlled one, thus matching qualitative pattern 
#5] 



dACC activity As described earlier, the dACC computes co-activation of both response units 
in FEF (i.e. when average activity is > 0.5) - a direct measure of conflict (or value of the alternative 
action to that initially considered; see above). Consequently, its activity (see figurej?^) follows a similar 
pattern as average FEF layer activity: conflict is present but resolved prior to responding in correct 
trials while conflict is present after responding in error trials. However, dACC does not get active in 
congruent trials, because it never shifts from one action to the other. 



This qualitative pattern of peak conflict activation before correct incongruent trials but after incor- 
rect incongruent trials matches event-related potentials (ERFs) commonly observed in human EEC 
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Figure 7: a) Averaged dACC activity (corresponding to conflict in FEF) in prosaccade and correct 
and incorrect incongruent trials. No conflict is present in congruent trials. During correct incongruent 
trials, conflict is detected and resolved before the response is gated. During incorrect incongruent 
trials, an incorrect response is made before conflict is detected, b) Activity recorded in monkey pre- 



SMA during the switch-task (reproduced from Isoda and Hikosaka (2007)). c) EEG recordings from 



the central scalp of humans during the Flanker task (reproduced from Yeung et al. (2004a)), thought 



to originate from dACC. The N2 and ERN component closely match our modeling results, replicating 
this aspect of the Yeung model. 



studies (see flgure^). The so-called error related negativity (ERN) which is measured after response 



errors whereas the so-called N2 potential is measured before correct high conflict responses (Falken- 



stein et al. 1991 Gehring et al. 1993). The idea that these two signals could merely represent 'two 



sides of the same conflict coin' and reflective of underlying dACC activity was flrst presented in the 



modeling work by Yeung and colleagues (Yeung and Cohen 2006 Yeung et al. 2004b) 



STN activity As noted in the model description, conflict detection in the dACC results in delayed 
(and more accurate) responding by recruiting the STN to prevent gating until conflict is resolved. 
Indeed, this mechanism is in part responsible for the rightward-shifted RT distributions in correct 
incongruent trials. Accordingly, this same pattern of increased activity before correct responses and 
increased activity after error responses can be observed in STN (see figure Isk). Again, this qualitative 



pattern was also found in STN recordings in monkeys by Isoda and Hikosaka ( 2008 1 (see figure [8] 
who showed that timing of STN firing relative to pre-SMA was consistent with communication along 
this hyperdirect pathway. 



The neurocomputational model of (Brown et al. 2004) interprets the role of STN differently. In 
their model, STN is activated by the output structure (FEF in their case) to lock out the infiuence 
of competing responses after a response has been selected. This is a critical difference to the account 
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Figure 8; a) Averaged activity of the model STN layer during prosaccade and correct and incorrect 
incongruent trials relative to response execution. During congruent trials STN units exhibit a small 
early increase in activity that subsides. Correct incongruent trials show increased activity early on in 
the trial which causes the conflict-induced slowing and prevents prepotent response gating. In error 
trials, this mechanism is triggered too late and the incorrect response gets executed, b) Electrophys- 
iological recordings of the monkey STN (reproduced from Isoda and Hikosaka ( 2008| )) on correct and 
incorrect switch trials and non-switch trials, c) Average activity of the STN layer of an alternative 
model in which STN is not excited by dACC but instead by saccadic output (SC in our model) as 
proposed by .Brown et al. (2004). This model does not predict differences between trial types. 



presented herein where STN plays a role in the selection of a response by raising the threshold prior 
to response selection, thereby delaying execution but increasing accuracy. To show explicitly how our 
model predictions can be qualitatively differentiated from this alternative model of STN function, we 
disconnected dACC inputs into the STN and instead allowed only the output structure (SC in our 
model) to project to it, so that STN function operates as it does in Brown et al (2004). As can be seen 
in figure [8]:, the activity pattern changes dramatically. Specifically, there is no more differentiation 
of activation patterns between the different trial types as is observed in our model and the empirical 



data (Isoda and Hikosaka 2008). Because STN only influences processing after response selection, it 



also does not lead to delayed responding or decision threshold adjustment. This qualitative difference 
in model predictions is fundamental and not subject to parameter tuning, as it reflects a distinct 
computational role for the STN. Although we focused on the Brown model for demonstration purposes 
here, other models of STN function with different connectivity would similarly not account for these 



data. For example, the biophysical model of Rubchinsky et al. (2003) assumes that STN neurons 



provide focused selection of a particular action (by disinhibiting SNr, taking the role of the direct Go 
pathway) while simultaneously inhibiting competing actions (by exciting SNr in other columns). This 
model cannot explain this activity pattern because co-activation of multiple cortical inputs does not 
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result in increased STN activity (see figure 6b in Rubchinsky et al. (2003)). 



Striatal activity Figure [9^ shows striatal activity in congruent and incongruent trials (column I 
and column II, respectively) for direct-path Go and indirect-path NoGo units (upper and lower rows, 
respectively). In each case, activity selective to the correct and error responses are color coded. The 



model closely captures the qualitative pattern across four cell populations (#7 1 identified in monkey 



dorsal striatum recordings during the antisaccade task (see figure^ and Watanabe and Munoz (2009); 



Ford and Everling (2009)). In particular, for congruent trials, correct-coding Go neurons gate the 



response while error-coding NoGo units suppress the alternative. In incongruent trials, Go neurons for 
the error-coding prepotent response are initially activated, but are then followed by increased activity 
of the corresponding NoGo population which then suppresses the initiated Go activity via NoGo—s-Go 



inhibitory projections (Taverna et al. 2008). Finally, the controlled Go-correct units are activated and 



an incongruent response is executed. Thus our model predicts that the pattern of electrophysiology 
observed in empirical recordings arises due to top-down cognitive control modulation of direct and 
indirect pathway neurons. 

Note again that we can distinguish our model's predictions from those of other models that omit 
the indirect pathway as a distinct source of computation (there are several) or from models that do 



include it but assign a different function. The neural network model of Brown et al. ( 2004 ) assumes the 



indirect pathway activation defers execution of the correct action plan until the time is appropriate. 
This would suggest that the executive control complex would activate NoGo units coding for the correct 
response, not the incorrect response as in our model. To demonstrate how this leads to qualitatively 



different patterns than is observed in our model and the data (see pattern ^7 and figure^) in which 



this alternative account is simulated in our model. (However, we note that the Brown et al model 
could potentially accord with our model in the sense that they also advocate a mechanism by which 
negative prediction errors drive learning in the NoGo cells, which after training on the AST may also 
produce the patterns we observe here given that the prepotent response would be punished). Similarly, 



the prominent model of Gurney et al. (20011 suggests that this pathway serves as a control pathway 
rather than providing negative evidence against particular actions as in our model, and it is unclear 
how this control function (while not disputed per se) would reproduce the patterns observed here. 



24 



Model 




Data 



Anti 



0.4 

8 " 

o 
Z 



0.2 



0.0 




C) 



Brown et al model 
Pro Anti 



Correct resp. 
coding 

Inconect resp. 
coding 



-600 -200 200 600-600 -200 200 600 -400 -200 200 400 -400 -200 -0 200 400 -600 -200 200 600-600 -200 200 600 

Figure 9: a) Averaged striatal activity during correct pro (first column) and incongruent trials (second 
column) in Go (first row) and NoGo (second row) neuronal populations. In each case, activity for 
correct (red) and error (blue) response coding units are shown separately. As described in the text, 
the Go units for the prepotent response become active early in the trial for both trial types, but in 
antisaccade trials these are followed by NoGo units which veto the Go activity and finally Go activity 
for the controlled response due to top-down DLPFC activity, b) Electrophysiological recordings of the 
monkey striatum (reproduced from Watanabe and Munoz (20091). The first row represents neurons 
coding corresponding to the executed response (i.e. Go neurons) and the second row represents neurons 
coding that suppress execution of the corresponding action (i.e. NoGo neurons), c) Alternative model 



simulating Brown et al. (2004) assumption that the indirect pathway acts to defer the execution of 
the correct response, rather than suppress the alternative response. Note predictions for Go pathway 
accord with those of our model and the data, but prediction of NoGo neurons differ. 

2.2 Global Response Inhibition 
2.2.1 Methods 

In SRITs the selectively inhibited prepotent response must be replaced with another, controlled re- 



sponse. Conversely, the stop-signal task (SST) requires outright response inhibition (e.g. Logan and 



Cowan 1984 Aron and Poldrack 2006[ [Cohen and Poldrack 2008 1 and is used to assess global in 



hibitory control (Aron 20111. Specifically, subjects are required to make press left and right keys in 
response to Go-cues appearing on a screen. On a subset of trials after the Go-cue has been presented, 
a stop-signal is presented after variable delay (i.e. stop-signal delay; SSD) instructing the subject to 
withhold responding. 

Here we show that our model can also simulate the SST after we included the right inferior frontal 



cortex (rIFG) with direct projections to STN (Aron et al. 2007a) see figure 10 Given the assump- 
tions of the race model (i.e., a race between Go and Stop processes), one can estimate the stop-signal 
reaction time (SSRT) by measuring the probability of successful inhibition at different SSDs. This 
inhibition function is then compared to the distribution of Go reaction times in non-stop signal trials. 



There are several extensive reviews of the SST (Verbruggen and Logan 2009b I, so here we focus on 



how our model captures the available evidence. Note that the SST typically refers to the task involv- 
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ing manual movements (and inhibition thereof), but a weh studied equivalent has been used in the 
oculomotor domain, where it is referred to as the 'countermanding task'. While the neuronal circuitry 
involved in Go-responding depends on the response modality, the neuronal circuitry involved in the 



global mechanism may be independent of the response modality ( Leung and Cai 2007 ) . 



Networks are presented with one of two input stimuli (left or right), represented by a column of 
four units each. As in prior simulations, prepotent responses are implemented by weights from the 
input units to the corresponding FEF response units, such that a left stimulus suggests a left response. 
On 25% of trials, a stop-signal is presented with variable delay (by activating devoted units in the 
sensory input layer). The stop signal units send excitatory projections directly to the rIFG layer. 



rIFG units in the hyperdirect pathway excite the STN (Aron et al. 2007a Neubert et al. 2010) and 



prevent striatal response gating, and therefore inhibit responding if the SC has not already surpassed 
threshold. In addition to this global rIFG-STN response suppression mechanism, the DLPFC com- 
bines the stop-signal input and the stimulus location to selectively inhibit the associated response via 
activation of the corresponding population of striatal NoGo units. Critically, this selective mechanism 
is slower but remains active after the STN returned to baseline and prevents subsequent responding. 
Thus, the model uses a fast, global but transient response inhibition mechanism and a slower, selective 



but lasting mechanism (Aron 2011). To estimate the SSRT, we use the dynamic one-up / one-down 



staircase procedure for adjusting the SSD (e.g. Logan et al. 1997 Osman et al. 1986). 



We tested the influence of rIFG lesions on the SSRT ( Aron et al. , 2004 ) by parametrically reducing 
the projection strength of rIFG to the STN. 



The selective norepinephrine (NE) reuptake inhibitor Atomoxetine increases NE release and im- 



proves stop-signal performance in animals, healthy adults and adult ADHD patients ( Chamberlain 



et al. 2007 2009). NE is hypothesized to adaptively change the activation gain of neurons in frontal 



cortex (Aston- Jones and Cohen, 2005[ ). We consequently tested the influence of decreasing the gain 
parameter in units of the frontal cortesj^ 



Finally, we simulated different motivational influences on stop-signal accuracy. Evidence for the 



neural underpinnings of motivational biases comes from an fMRI study by Leotti and Wager (2010), 



who reported that subjects instructed to focus on speed instead of accuracy exhibited a greater increase 



^Gain modulates how step-like the activation-dynamics of units are in relation to their input activity. Low gain leads 
to linear activation dynamics while high gain levels make a unit respond in a binary-like fashion. 
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Figure 10: Extended neural network model including rFIG during stop-signal trials. (1) Left input 
stimulus activates (2) left-coding FEF response units and (3) initiates gating via striatum (similar to 
pro-saccade trial in a). After a delay, (4) the stop-signal is presented which activates (5) rIFG, which 
in turn (6) transiently activates the STN and finally (7) the whole SNr to globally prevent gating. 
Note, that DLPFC is beginning to get active to initiate selective response inhibition via striatal NoGo 
units. 
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in activations in brain regions associated with response facilitation, including the FEF and the striatum. 
Conversely, when instructed to focus on accuracy, subjects exhibited greater activity in IFG regions 
associated with response inhibition. We thus simulated these activation patterns to account for speed- 
accuracy tradeoff in a similar manner as in the antisaccade simulations. In the speed-condition, 
we manipulated the strength of FEF to striatum connections due to evidence that frontostriataal 



connectivity is enhanced under speed emphasis (Forstmann at al. 2008, 2010 Mansfield et al. 2011) 



Conversely, in the accuracy condition we increased baseline excitatory input to rIFG, allowing it to 
be more excitable and hence facilitating STN recruitment. This simulation approximates the effect 
of a putative PFC rule based representation to focus on accuracy. Recent data supports the notion 
that the (right) STN, which receives input from rIFG, shows increased excitability associated with an 



increased response caution during accuracy focus (Mansfield et al. 2011). 



2.2.2 Results 

As with the SRITs above we extracted a list of key qualitative results from the literature we use to 
evaluate the fit of our model. 



7^1 The probability of inhibiting a response decreases monotonically as SSD increases (Verbruggen 



and Logan 2008). 



#2 Error responses that escape inhibition are, on average, faster than Go responses on no-stop-signal 
trials. However, while the distributions begin at the same minimum value, the responses that 



escape inhibition have a shorter maximum value ( Verbruggen and Logan 2008 1 . 



#3 STN neurons are excited to stop signals but show little differentiation between stop-signal inhi- 



bition and stop- respond error trials ( Aron et al. 2007a I . Contrary, downstream SNr neurons are 



excited in correct trials but are disinhibited during error trials (Schmidt et al. 2012) 



#4 SEF neurons are activated in stop-signal and stop-response trials after SSRT and can thus not 



contribute to successful stopping (Stuphorn et al. 2000). 



Behavior To illustrate the staircase procedure, figure 11(a) shows an example trace of how SSDs are 
adjusted to assess 50% stop-signal accuracy. As can be seen, the network with rIFG lesion is impaired 
at stopping and requires shorter SSD on average to inhibit successfully. 



As can be seen in figure 11(b) the inhibition function resulting from testing the neural network 
systematically with different SSDs reveals a monotonically decreasing probability of correctly stopping 
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Figure 11: a) Progression of the staircase procedure for manipulating SSD in networks with reduced 
rIFG-STN connectivity. Trial number is plotted on the x-axis and the stop-signal delay (SSD) in ms 
(converted from simulator time) is plotted on the y-axis. If a response is successfully inhibited on 
stop-signal trial, the SSD is increased by 20 ms to make it harder. If a response is erroneously made 
on a stop-signal trial, the SSD is decreased by 20 ms. Networks without lesion are highest in general 
representing the most effective Stop-process that is able to withhold responses even when the SSD is 
quite long, b) Inhibition function of the neural network model in the stop-signal task. The model is 
tested on systematically varying levels of stop-signal delay (SSD) in ms and the proportion of correctly 
inhibited trials is plotted along the y-axis. 



(qualitative pattern ^1 ) 



Cumulative RT distributions of Go and non-canceled Stop trials are presented in figure [T2| Both 



distributions match closely up until SSD+SSRT (qualitative pattern #2) suggesting that both are 
generated by the same process. 

Different modulations affect GoRT and SSRT in different ways (figures [13(a)] and [13(b)]). While DA 
manipulations certainly speed GoRT, SSRT remains largely unaffected. On the other hand, when the 
network is tested with reduced gain (simulating low NE levels), or has lesions to either STN or rIFG, 
it exhibits SSRT deficits (increases). Finally, simulated accuracy emphasis results in slowed Go RT 
but faster SSRT (more effective inhibition). The pattern that emerges from these results is that SSRT 
is changed by modulations of parameters that are part of the global inhibitory pathway: rIFG and STN. 



Neurophsyiology To assess the neural correlates of stopping behavior in our model we analyzed 
STN and SNr activity aligned to stop-signal onset. As can be seen in figure [Mj there is little differ- 
entiation between stop-signal inhibition and error trials while SNr units show a marked dip in error 
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Figure 12: a) Cumulative reaction time distributions of the neural network model and from a monkey 
experiment, b) Cumulative reaction time distribution from a monkey experiment for comparison. 
Reproduced from (Lo et al. 20091. The solid red line denotes mean stop-signal delay (SSD); the 



broken red line denotes stop-signal reaction time (SSRT) offset at SSD. The broken blue horizontal 
line represents 50% stopping accuracy. Note that the response distribution sums to the response 
probability - not necessarily to 1. 
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Figure 13: a) Mean RTs in ms ± SEM (converted from simulator time) for Go trials under different 
modulations (see text), b) Mean SSRTs in ms ± SEM (converted from simulator time) under different 
modulations (see text). 
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a) Model: Go 



b) 



Model: SNr 



c) Model: STN 



d) 
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Figure 14: Average activity aligned to stop-signal onset for inhibited and error stop-signal trials, a) 
Striatal Go-neuronal activity, b) Substantia nigra pars reticulata activity, c) Subthalamic nucleus 
activity, d) Activity of the executive control complex consisting of DLPFC, SEF and pre-SMA. 



trials that is less pronounced in inhibition trials (qualitative pattern =^31. 

We moreover analyzed the activity pattern of our executive control complex which consists of 



DLPFC, SEF and pre-SMA. As can be seen in figure 14 activation is observed in stop-signal trials 
(both stop-respond and successful inhibitions) only after SSRT and could thus had no influence on the 



stopping (qualitative pattern ^3 ) . This result implies that global stopping to salient stop signals is 



most likely driven by the fast stop process along the rIFG-STN hyperdirect pathway. We ascertain that 
executive control processes are delayed relative to this global stopping mechanism, and may participate 
in selective response inhibition (and in the stop-change task, activation of the correct response) after 
the global response pause has passed. 



3 Discussion 



The interaction between executive control and habitual behavior is a central feature of higher-level 
brain function, and plays a role in various domains from cognitive psychology (under the rubric of 



"system 1" vs. "system 2"; (Evans, 20051) to machine learning (model-free vs. model based control 



(Daw et al. 2005)). At the core of this interaction is a mechanism that allows executive control to 



override the habitual system and guide action selection. A multitude of psychological cognitive tasks 
have been used to probe the nature of this interaction. The stop-signal task requires outright stop- 



ping of a response already in the planning stage. The antisaccade (Hallett 19791, Simon (Simon 



19691, and saccade override (Isoda and Hikosaka 2007 2008) tasks all involve inhibition of a prepo- 



tent action together with initiation of an action incompatible with the prepotent one. Despite the 
apparent behavioral simplicity of these tasks, various lines of research have revealed a highly complex 
and tightly interconnected brain network underlying response inhibition consisting of frontal areas 
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including DLPFC, SEF, pre-SMA, FEF, rIFG, and dACC and basal ganglia structures including the 
striatum and STN. 



We presented a dynamic neural network model of selective and global response inhibition which 
provides a description of the distributed computations carried out by individual brain regions and 
neurotransmitters. The complexity of this model is grounded by well established neuroanatomical and 
physiological considerations, and accounts for a wealth of key data including electrophysiology, psychi- 
atric and pharmacological modulations, behavioral, lesion and imaging studies. Moreover, this model 
is constrained (i) by using a single parameterization across all simulations of intact function and (ii) by 
the multitude of qualitative results from different levels of analysis it is required to reproduce. Although 
we used one parameterization across the intact model simulations, we also generalized the functionality 
via systematic manipulations across a range of parameters. In other work (Wiecki & Frank, in prep), 
we have shown that the emerging fundamental computational properties of this complex system as a 
whole are captured by analysis using a modified drift diffusion model, in which distinct mechanisms 
within the neural model (e.g., STN projection strength, DLPFC speed) are monotonically related to 
high level decision parameters (e.g., decision threshold, and drift rate of the executive process). 



3.1 Selective Response Inhibition 

In our SRIT simulations, the model assumes that prepotent, reflexive actions such as a saccade to an 
appearing stimulus (e.g. a prosaccade) are selected via the cortico-cortical route and swiftly gated 
by the BG. An abundance of data supports the general involvement of the BG in saccade generation 



Hikosaka et al. 


2000 Hikosaka 


1989 


Hikosaka and Wurtz 


1986 



cognitive control system not only represents the task rules needed to respond appropriately (e.g. in 
DLPFC), but also incorporates a downstream mechanism in dACC-STN to detect when these rules 
indicate an alternative action than was originally initiated. Thus our model synthesizes the popular 



account of dACC in terms of response conflict (Botvinick et al. 2001 1 with recent studies suggesting 



that dACC rather reflects the value of the alternative action (Rolling et al. 2012 1. Moreover, via 
the hyperdirect pathway to the STN, this mechanism serves to transiently increase the BG gating 
threshold to prevent prepotent actions from being facilitated and allows more time for the controlled 
PFC-striatal mechanisms to selectively suppress this response and to facilitate appropriate alternative 



courses of action. It has also been shown that the SEF, FEF (Munoz and Everhng 2004), dACC 



( [Botvinick et al.[|2004| ), pre-SMA ( [Isoda and Hikosaka[ [2007| ) and STN ( [Isoda and Hikosaka[ [2008 ) 
are involved in detecting conflict between a planned response and the current rule, and for switching 
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from an automatic to a volitional response (e.g., antisaccades) . 



To detect conflict between reflexive and controlled responses, the system needs to be able to com- 
pute the correct identity of the controlled response itself. In the model, the DLPFC integrates task 



instructions and current stimulus location and forms a conjunctive rule representation (Wallis and 



Miller 2003 Bunge and Wallis 2008) that then provides evidence for the associated controlled re- 



sponse via its projection to the FEF, and further biases the gating of this response (and the selective 
suppression of the reflexive response) via striatum. We demonstrated that this is a necessary condition 
for our model by showing that a model with faster integration speeds fails to account for key behavioral 
patterns. 



Thus it should be clear that compared to a congruent response, an incongruent response should 
(i) be more prone to error because it depends on successful inhibition of prepotent actions which 
may be close to threshold by the time conflict is detected and (ii) take longer due to (iia) additional 
computation needed for the DLPFC to perform the requisite vector inversion (activation of correct 
rule representation among multiple competitors based on an integration of input and instruction), and 
(iib) the delay in commitment to a response resulting from the increase in decision threshold along the 
hyperdirect pathway. 



Early cognitive models of interference control assumed a dual-route mechanism for action selection, 



including an automatic response route and a volitional one (Kornblum et al. 1990 Eimer 1995 



DeLiang et al. 2005 Ridderinkhof 2002). This model was extended to include selective suppression 



of the automatic response by the volitional response mechanism (i.e. the activation-suppression model 
(Ridderinkhof 2002 Ridderinkhof et aL||2011 )). Our model shares these attributes but makes two 
crucial contributions to this discussion: (i) strong predictions on the neural correlates of these abstract 
cognitive processes, and (ii) a raise in decision threshold requiring more evidence to gate any response. 
This latter mechanism may not only be adaptive as a fast route to prevent gating of prepotent actions, 
but could also serve to increase the likelihood that the alternative action selected is the most accurate 
(particularly when there may be more than one, as is often the case in more realistic executive control 
scenarios than those typically studied in simple response inhibition tasks). 
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3.1.1 Response time distributions and errors: Neural underpinnings 

At the behavioral level, our intact model reproduces the same patterns found empirically - networks 
made more errors (see figure 3(a)) and were in general slower (see figure [3(b) [ ) on incongruent trials 
compared to congruent trials (e.g. Reilly et al. 2006 Harris et al.[ 2006[ [Reilly et"aL 2007[ McDowell 
|et al.[ 2002 1 . Incongruent errors were more likely to occur when networks responded fast (see figure 
in [5(a)1 and [Ridderinkhof et al.| ( [20TT| ). These errors result primarily from variance m the speed of 
cognitive control (DLPFC), but also in the prepotent response (in some trials gating is faster than 
others) and in the inhibition process (in some trials the hyperdirect pathway and/or striatal NoGo 
process is slower). Moreover, reduced DLPFC connectivity also degrades accuracy on incongruent 
trials, mirroring the empirical performance degradation in antisaccade tasks during development asso- 



ciated with reduced DLPFC connectivity (Hwang et al. 2010). A more explicit investigation into the 



dynamics of these processes comes from the simulated electrophysiology across brain regions and trial 
types. 

3.1.2 Conflict- and error-related activity: relation to existing models 

The Error Related Negativity (ERN) is an event-related potential associated with errors made in 



forced-choice reaction time tasks (Falkenstein et al. 1991 Gehring et al. 1993). The ERN reaches its 



peak within 100 ms after the erroneous response. Using a connectionist model, Yeung and coUegues 
hypothesize the ERN to reflect conflict between the executed, erroneous response and the still- evolving 



activation of the correct response (Yeung and Cohen 2006 Yeung et al. 2004b I. Thus, the error detec 



tion mechanism reflects an internal correction of the executed response, leading to a transient period 
of response conflict. According to this same framework, a similar potential should be observed in high 
conflict trials before correct responses, when conflict is resolved prior to responding. These authors 
indeed reported that the N2 potential exhibited just this profile and argued that it reflected the same 
underlying conflict mechanism in the dACC. 



Our dACC node exhibits the same qualitative pattern of increased activity (i) before correct incon- 
gruent responses, (ii) after incorrect incongruent responses and (iii) baseline activity during congruent 
responses. However, this pattern is not unique to ERPs thought to originate from dACC, but is also 



found in electrophysiological recordings in pre-SMA, SEF (Emeric et al. 2010) and STN (Isoda and 



Hikosaka 2008). Our model provides an explicit framework that recapitulates these effects and ex- 



plores their influences on behavior. Together, these dynamics accord with our earlier assertion that 
our model synthesizes the conflict model with the notion that the dACC reflects the value of the alter- 



34 



native action: this network only becomes activated when the ahernative action is deemed to be more 
correct than the prepotent one. This process occurs either prior or following response execution (as 
in the conflict monitoring account), but must always occur after the initial activation of an incorrect 
(often prepotent) response (not specified by the conflict account but consistent with the alternative 
action value account). To more formally describe the computational dynamic implicated, we devised 
a modifled drift diffusion model which explicitly incorporates this reversal in evidence. 



3.2 Global Response Inhibition 

By adding a single rIFG layer to our model we generalized our model to capture data from global re- 
sponse inhibition tasks such as the SST. As we demonstrated above, this model recovers key qualitative 
behavioral patterns reported in the literature. Moreover, model neurophysiology revealed interesting 



similarities to recent rat electrophysiological recordings in the SST (Schmidt et al. 2012). Specifically, 
while STN activity surges in response to the stop signal to the same extent regardless of whether the 
response is successfully inhibited or not, activity in the SNr strongly differentiates between these trial 
types. During errors, the striatal Go signals were potent and early enough to inhibit SNr activity in 
spite of the STN surge. These results suggest that the source of response inhibition errors is variance 
in the Go process, but that the duration of the stop-process is rather fixed. This conceptualization 



matches closely with the interactive horse-race model (Verbruggen and Logan 2009a). Here, we hy- 
pothesize that the critical point of interaction between the two processes is the SNr. 



Why did we add an rIFG layer given that our initial model already contained an executive control 
complex including DLPFC? As described above, rIFG and STN involvement in the SST is well estab- 
lished, and moreover, simulations showed that the activations in our executive control complex needed 
to account for SRITs was too slow to account for global response inhibition needed in SST. Never- 
theless, the nature of the (potentially separable) mechanisms engaged for detecting when inhibitory 
control is necessary, and how it should be implemented, remains largely elusive. In particular, the role 



of rIFG is actively debated. Some studies specifically implicate the rIFG in response inhibition ( Ver- 



bruggen et al. 2010 Aron et al. 2003 Chambers et al. 2007 Sakagami et al. 2001 Xue et al. 2008), 



whereas others report rIFG activity in tasks lacking pure response inhibition demands, suggesting that 



it is more involved in monitoring or salience detection (Sharp et al. 2010 Verbruggen et al. 2010 



Hampshire et al. 


2010 


Fleming et al. 


2010 


Chatham et al. 


2012 


Munakata et al. 


2011 



unifies these two seemingly opposing views by arguing that the rIFG in fact detects salient events and, 
via downstream processing, engages a stopping mechanism whether or not it is required by the task 
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rules. In both the stop-signal and stop-change task, subjects have to detect an infrequent signal which 
tells them to update their current action plan. We argue that these signals are salient events and, 
via noradrenergic modulation, enhance processing in the rIFG which, in turn, causes an orienting or 



circuit breaker response by activating the STN (Swann et al. 2011a) to pause response selection. This 
pause enables the volitional DLPFC based response selection mechanism to take control and either 
inhibit a specific response (as in the stop-signal task) or initiate a new response (as in the stop-change 
task). This theory of a rIFG triggering a global response-pause is supported by rIFG involvement in 



the oddball task ( Stevens 2000 Huettel and McCarthy 2004 ) which requires no behavior adaptation 



whatsoever, yet still causes response slowing (Barcelo et al. 2006 Parmentier et al. 20081. Indeed, 
in many of the above-reported studies in which rIFG is activated under conditions of monitoring or 
saliency, when they have been reported, subject RTs were nevertheless delayed despite no overt in- 



hibitory demands (Sharp et al. 2010 Fleming et al. 2010 Chatham et al. , 2012 1. 



3.3 Different forms of response inhibition 



Inhibitory control can be issued globally or selectively ( Aron and Verbruggen 2008 Aron 2011 1. The 
brain seems to revert to a global inhibitory mechanism when unexpected events occur that require 
quick response adaptation (e.g., stop-signals), and to a selective inhibitory control mechanism when 



response inhibition can be prepared (Greenhouse et al. 2011 Hu and Li 2011). We propose that 



selective inhibition of the prepotent response is initiated by the DLPFC and implemented via the 



indirect corticostriatal NoGo pathway (Zandbelt and Vink 2010 Watanabe and Munoz 2009 2010 



Hu and Li 2011 Jahfari et al., 2011). Global response inhibition on the other hand is driven by a 



salience detection mechanism implemented in the rIFG which directly projects to the STN to inhibit 
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In addition to the selectivity of inhibitory control, differences exist between proactive and reactive 
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2011). Our modeling work suggests multiple possible sources for proactive control. Speed-accuracy 
adjustments are implemented by increasing functional connectivity between frontal motor regions 
and striatum to decrease the decision threshold under speed emphasis (see figure |5(b)[ |5(c) and Lo 



and Wang 


( 


2006 


); 



caution by increasing baseline rIFG activity to prime saliency detection and slow responding via 
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the rIFG-STN hyperdirect pathway (see figure |13(b)| . Interestingly, while FEE— T^striatum functional 
connectivity influence speed and accuracy in our SRIT simulations, SSRT in the stop-signal task is 
unaffected by this modulations and is only improved by an increase in tonic rIEG activity. This 
suggests that proactive control in form of mere response slowing is uneffective in reducing SSRT - the 
staircase procedure adapts to slower overall responding - but that enhanced attentional monitoring 
has preferential influence on global inhibitory control. In other words, although all these mechanisms 
can lead to adjustments in decision threshold, only those associated with active engagement of the 
stop process will facilitate inhibitory control per se. If confirmed, this result may have implications 
for refining therapy of inhibitory control disorders like addiction, obesity and OCD. Nevertheless, it 
remains important to emphasize that the striatal NoGo pathway is also thought to help to prevent the 
proactive selection of maladaptive responses. 



3.4 Multiple mechanisms of response threshold regulation in fronto-basal- 
ganglia circuitry at different time scales 

Different mechanisms in our neural network influence the gating threshold for initiating motor re- 
sponses at distinct time scales, and modulated by distinct cognitive variables. First, the strength of 
cortico-striatal projections regulate the ease with which cortical motor plans can be gated by the BG, 



allowing for speed emphasis in the speed-accuracy tradeoff (see figure 5(c) ). This aspect of our model 



is quite similar to the model of Lo and Wang ( 2006 ) and was subsequently corroborated by Forstmann 



at al. (2010). Our model converges on the same conclusion but extends this view by showing that 



gating threshold is also more dynamically regulated on a shorter time-scale by (i) motivational state 
(changes in DA levels, which are modulated by reinforcement and also facilitate striatal Go signals); 
and (ii) response conflict and saliency (via the hyperdirect pathway, making it more difficult or Go 



signals to drive BG gating (Jahfari et al. 2011)). Moreover, STN efficacy in the neural model is pos- 



itively correlated with increases in estimated decision threshold (Ratcliff and Frank 2012 1. Evidence 
for conflict-induced decision threshold adjustment via the hyperdirect pathway has been recently de- 



scribed in a reinforcement-based decision making task (Cavanagh et al. 2011). Increases in frontal 
EEG activity during high conflict decisions were related to increases in decision threshold estimated 
by the drift diffusion model. Intracranial recordings directly within the STN also revealed decision 
conflict-related activity during the same time period and frequency range as observed over frontal 



electrodes (see also Zaghloul et al. (2012)). Moreover, disruption of STN function with deep brain 
stimulation led to a reversal of the relationship between frontal EEG and decision threshold, without 
altering frontal activity itself. These data thus support the notion that frontal-STN communication 
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is involved in decision threshold adjustment as a function of conflict. Similarly, proactive preparation 
to increase decision threshold in the stop signal task when stop signals are likely is associated with 
hyperdirect pathway activity (Jahfari et al. 2012). 



In our neural models, conflict-related STN activity subsides with time (see flgure|8]), due to resolu- 
tion of conflict in FEF/ACC, feedback inhibition from GPe, and neural accommodation. Thus a more 
refined description of this transient STN surge is that it initially increases the decision threshold (more 
so with conflict), followed by a dynamic collapse of the decision threshold over time. Indeed, a recent 



multilevel computational modeling and behavioral study by Ratcliff and Frank (2012) supported this 
idea by showing that a collapsing threshold diffusion model provided good fits to both the BG model 
and to human participant choices in a reinforcement conflict task. Moreover, the temporal profile 
of the best-fitting collapsing threshold corresponded well to the time course of the collapse in STN 
activity across time. 



3.5 Psychiatric disorders and differential effects of dopamine and nore- 
pinephrine 

Abnormal striatal DA signaling is hypothesized to be at the core of many disorders, including PD 



(Bernheimer et al. 1973), SZ (Breier et al. 1998) and ADHD (Casey et al. 2007 Frank et al. 2007b) 



Intriguingly, all of these disorders are linked to response inhibition deficits in the stop-signal task. Our 
earlier BG models have successfully accounted for a wide variety of findings associated with striatal DA 



manipulations across reinforcement learning and working memory tasks (for review, Wiecki and Frank 



2010). Yet, we found here that striatal DA manipulations, while affecting overall RT, had negligible 



effects on response inhibition deficits as assessed by SSRT (see figure [l3(b)[ ). This prediction converges 



with recent evidence (reviewed in, Munakata et al. 2011 ) showing that levodopa, a drug that increases 
DA levels in striatum ( [Harden and Grace 1995), had no influence on SSRT in PD patients (Obese 
eral|[20lla|b |. 



This lack of DA effect raises the question of the source of the response inhibition deficits in the 
aforementioned disorders. One conspicuous candidate is abnormal NE functioning as suggested by 



evidence in both ADHD (Faraone et al. 2005 Ramos and Arnsten 2007 Frank et al. 2007c) and 



PD (Farley et al. 19781. In our simulations, NE modulation influences SSRT via its gain- modulatory 



effects in rIFG (Aston- Jones and Cohen 2005). Additional support for this account comes from 
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pharmacological experiments using the selective norepinephrine reuptake inhibitor atomoxetine, which 



improves response inhibition performance in animals, healthy adults and ADHD patients ( Chamberlain 



et al. 2007 2009). Moreover, IMRI analysis revealed that atomoxetine exerted its beneficial effects via 



modulation of rIFG ( Chamberlain et al. 2009 ), providing additional support for the model mechanisms. 
Finally, this highlights an alternative source for response inhibition deficits observed in PD patients 



previously linked to DA dysfunction (see Vazey and Aston- Jones (2012) for a review highlighting the 
importance of aberrant NE signaling in cognitive deficits of PD patients). 



4 Limitations 

Despite our model's success in reproducing and explaining a wide array of data and offering potential 
solutions for long standing issues in the field, we certainly acknowledge that there are many errors 
of omission and - although we did not include any biological features that are unsupported by data 
- perhaps some errors of commission. We note however note that most of our assumptions and 
simulations are largely orthogonal to each other. Thus, each aspect of the model is falsifiable on its 
own, without necessarily falsifying other aspects. We discuss a few salient limitations below; it is by 
no means exhaustive. 



4.1 Specificity of PFC regions and function 

While the BG of our neural network model is fairly concrete and solidly grounded on ample anatomical 
electrophysiological, and functional evidence, the individual contributions of frontal regions including 
DLPFC, SEF, pre-SMA, FEF and dACC are not as well established currently. For example, we 
identified an executive control network in our model consisting of DLPFC, SEF and pre-SMA. The 
task rules and necessary motor commands to follow them are implemented by hard-coded input and 
output weight patterns of its extended network (i.e. sensory input, instruction, FEF and striatum). 
This implementation short-circuits a lot of the computational complexities the biological system has 
to solve; (i) the executive controller has to selectively retrieve the appropriate rule for the current 
trial from short or long-term working memory; (ii) integrate the sensory evidence to compute the 
correct response (e.g. via vector inversion); (iii) compute the necessary motor sequences to perform 
the correct action; and (iv) identify incorrectly activated prepotent responses and selectively suppress 



them. While neural network models with a more detailed representation of PFC exist (e.g. O'Reilly 



and Frank, 2006) in which rule-like representations can develop through experience, how exactly the 



necessary computations can be implemented dynamically is as-of-today a still unresolved question. 
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Critically, our focus in this work was on how PFC and BG interact when inhibitory control is 



required by extending the detailed BG model by Frank (2006). We also account for some electrophysi- 
ologcal data in frontal cortex, while acknowledging that there is still some uncertainty in the respective 
roles of these areas and their interactions which will be open for revision as more data become available. 



4.2 Learning 



Previous BG models explored the role of DA in feedback driven learning (Wiecki and Frank 2010). As 
humans (but not monkeys) are able to perform this task without learning, we chose to remain agnostic 
about the type of learning that takes place prior to performing the task. We thus hard-coded task 
rules into the model. An additional driving factor is the lack of published reports on specific learning 
phenomena in the SST and AST. 



5 Conclusions 

We presented a comprehensive, biologically plausible model of global and selective response inhibition 
which takes known properties of the neuronal underpinnings into account and tries to link them with 
results from cognitive science, electrophysiology, imaging studies and pharmacological experiments. 
Here, we showed that augmenting our previously described BG model with the addition of the FEF, 
DLPFC, and rIFG allows us to simulate control over prepotent responses and to capture a wealth of 
data in this domain across multiple levels of analysis. We furthermore provide multiple mechanisms 
that can lead to disruptions in inhibitory control processes and which have implications for interpre- 
tation of data from patients with psychiatric disorders such as SZ and ADHD. Our model shows that 
the observed deficits in inhibitory control paradigms do not necessarily have to reflect dysfunctional 
response inhibition per se but could be due to other factors such as salience, conflict detection and/or 
motivation, and related to distinct neural mechanisms. 
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7 Appendix 
7.1 Software 

The model and the Python scripts are available at http://ski.clps.brown.edu/BG_Projects/. 



7.2 Implementation details 



Like the original Frank (2006) model, this model is implemented in the Emergent neural modeling 



software framework (Aisa et al. 2008), which can be downloaded here: 
http : //grey. Colorado . edu/emergent/index .php/Main_Page. 

Emergent measures simulator time in cycles. Here, we convert this time to ms by multiplying cycles 
by 4 to roughly match behavioral and electrophysiological data. 



Emergent uses point neurons with excitatory, inhibitory, and leak conductances contributing to 
an integrated membrane potential, which is then thresholded and transformed via an ^^qj-j- sigmoidal 
function to produce a rate code output communicated to other neurons (discrete spiking can also be 
used, but produces noisier results). 

The membrane potential Vm is a function of ionic conductances g with reversal (driving) potentials 
E as follows: 

AK„(i) =TY,9c{tMEc - V^{t)) (1) 

c 

with 3 channels (c) corresponding to: e excitatory input; 1 leak current; and i inhibitory input. 
Following electrophysiological convention, the overall conductance is decomposed into a time- varying 
component gdt) computed as a function of the dynamic state of the model, and a constant that 
controls the relative influence of the different conductances. The equilibrium potential can be written 
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in a simplified form by setting tlie excitatory driving potential (i?e) to 1 and the leak and inhibitory 
driving potentials (£"/ and Ei) of 0: 



K^r = _ ^"t — = (2) 

ffeffe + gigi + gi9i 

which shows that the neuron is computing a balance between excitation and the opposing forces of 
leak and inhibition. This equilibrium form of the equation can be understood in terms of a Bayesian 



decision making framework (O'Reilly and Munakata 2000). 



The excitatory net input/conductance ge{t) or rjj is computed as the proportion of open excitatory 
channels as a function of sending activations times the weight values: 

Vj = 9e{t) = {XiWij} = - '^XiWij (3) 

i 

The inhibitory conductance can either be computed by the kWTA function described in the next 
section or by modeling inhibitory interneurons. Leak is a constant. 

Activation communicated to other cells (yj) is a thresholded (Q) sigmoidal function of the membrane 
potential with gain parameter 7: 

2/.W = 7 V (4) 

1 



7[v'™(i)-e]+ 

where is a threshold function that returns if xO and x if xO. To avoid dividing by we assume 
yj{t) = if it returns 0. This activation is subject to scaling factors (wt_scale.abs and wt_scale.rel) 
which modify how much impact the projections have on the post-synaptic neurons. 

7.3 Inhibition within and between layers 

Inhibition between layers (i.e. for GABAergic projections between BG layers and striatal inhibitory 
interneurons) is achieved via simple unit inhibition, where the inhibitory current for the unit is deter- 
mined from the net input of the sending unit. For within layer lateral inhibition (used here in premotor 
cortex) , Leabra uses a kWTA (k- Winners- Take- All) function to achieve inhibitory competition among 
neurons within each layer (area). The kWTA function computes a uniform level of inhibitory current 
for all neurons in the layer, such that the k + 1th most excited unit within a layer is generally below 
its firing threshold, while the kth is typically above threshold. Activation dynamics similar to those 
produced by the kWTA function have been shown to result from simulated inhibitory interneurons 



that project both feedforward and feedback inhibition (O'Reilly and Munakata 20001. Thus, although 
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the kWTA function is somewhat biologieally implausible in its implementation (e.g., requiring global 
information about activation states and using sorting mechanisms), it provides a computationally ef- 
fective approximation to biologically plausible inhibitory dynamics. kWTA is computed via a uniform 
level of inhibitory current for all neurons in the layer as follows: 

9i = 9k+i + Qidk - 9k+i) (5) 

where Ogl (0.25 default) is a parameter 6 for setting the inhibition between the upper bound of 
Qk and 9 . These boundary inhibition values are the lower bound of gk+i computed as a function of 
the level of inhibition necessary to keep a unit right at threshold: 

9i=9f+i+q{9f-9f+i) (6) 

In the basic version of the kWTA function, which is relatively rigid about the kWTA constraint 
and is therefore used for output layers, g^* and .gf^i '^'''-^ ^"^^ ^'^ ^^'^ threshold inhibition value for the 
kth and k+lth most excited neurons, respectively. Thus, the inhibition is placed exactly to allow k 
neurons to be above threshold, and the remainder below threshold. For this version, the q parameter 
is almost always .25, allowing the kth unit to be sufficiently above the inhibitory threshold. 

The premotor cortex uses the average-based kWTA version, gf is the average gf value for the 
top k most excited neurons, and gf_^_i is the average of gf for the remaining n — k neurons. This 
version allows for more flexibility in the actual number of neurons active depending on the nature of 
the activation distribution in the layer and the value of the q parameter (which is typically .6), and is 
therefore used for hidden layers. 

Hysterisis and Accommodation 

hit) = ga{t)g-a{VUt) - Ea) (7) 



hit) = gh{t)g-h{Vm{t) - Eh) (8) 

Efi is excitatory; Ea inhibitory. 

ga and gh are time-varying functions that depend on previous activity, integrated over different 
time periods. 
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9a{t) = { (9) 

.ga(i - 1) + dtg^{Q - - 1)); if(6a(i) = 6^) 

7.4 Computation of conflict 

dACC activity is the Hopfield energy of pre-SMA: 

dACCact = FEFieft,,, * FEFnght,,, (10) 
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